Privacy screen

ABSTRACT

A first device coupled with a first display and an image sensor receives output data from a second device having a second display different from the first display. The output data represents content displayable by the second device on the second display. The first device determines, using the image sensor, a position of the second display relative to the first device and causes the first display to display content based on the output data received from the second device and the determined position of the second display relative to the first device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/561,043, titled, “Privacy Screen,” filed Dec. 23, 2021, which is acontinuation of U.S. patent application Ser. No. 16/833,230, now U.S.Pat. No. 11,243,734, titled “Privacy Screen,” filed Mar. 27, 2020, whichis a continuation of International Application No. PCT/US2018/052868,titled “Privacy Screen for Computer Simulated Reality,” filed Sep. 26,2018, which claims priority to U.S. Provisional Patent Application No.62/566,187, titled “Privacy Screen for Virtual and Augmented Reality,”filed Sep. 29, 2017, the contents of each of which are incorporated byreference in their entireties.

FIELD

The present disclosure relates to the display of digital content ofexternal devices in computer simulated reality.

BACKGROUND

In some circumstances, a user of an electronic device, such as acomputer, phone, or other computing device with a display, will want toprevent other people from seeing content displayed on a screen of thedevice. For example, a user may wish to use a portable device in apublic café or on an airplane to work on private documents. Traditionalsolutions include a physical screen that is placed in front of thedisplay and that restricts the angles at which light emitted from thedisplay can be seen.

BRIEF SUMMARY

Traditional privacy screens as described above have several undesirablecharacteristics. For example, traditional privacy screens attenuate someof the light in the direction perpendicular to the display, whichreduces the brightness perceived by the user. To compensate, a user mayremove the privacy screen when privacy is not a concern, in which casethe user has to store, transport, and re-install the privacy screen.Alternatively, a user may increase the brightness output of the displayto compensate for the amount of light blocked by the privacy screen,which reduces the battery life of the device. Also, by blocking light innon-perpendicular directions, the privacy screen restricts the angles atwhich the user can use the device. Yet, a traditional privacy screenstill may not block all light emitted at non-perpendicular angles,thereby allowing others that are not directly in front of the devicesome visibility to the content on the display. Finally, a traditionalprivacy screen does not prevent displayed content from being observableby other individuals that are also positioned directly in front of thescreen (e.g., directly behind the user).

The techniques described below more effectively and efficiently protectthe content of an electronic device by restricting non-users from beingable to observe the content displayed by the device. As can be seen inthe discussion below, such techniques may have the additional advantageof conserving power and increasing the time between battery charges forbattery operated devices.

In some embodiments, a method is performed using a system that includesa first device with a first display and a second device with a seconddisplay different than the first display. The method includes:producing, by the first device, output data responsive to input;determining if the first device is operating in a first mode or a secondmode; if the first device is operating in the first mode, displayingcontent on the first display in accordance with the output data; if thefirst device is operating in the second mode, refraining from displayingcontent on the first display in accordance with the output data, andtransmitting the output data to the second device; determining, using animage sensor, a position of the first display relative to the seconddisplay; and displaying, on the second display, content in accordancewith the output data and the determined position of the first displayrelative to the second display.

In some embodiments, a system includes, a first device with a firstdisplay and a second device with a second display different than thefirst display. The first device is configured to: produce output dataresponsive to input; determine if the first device is operating in afirst mode or a second mode; if the first device is operating in thefirst mode, display content on the first display in accordance with theoutput data; and if the first device is operating in the second mode,refrain from displaying content on the first display in accordance withthe output data, and transmit the output data to a second device. Thesecond device is configured to: display, on the second display, contentin accordance with the output data and a determined position of thefirst display relative to the second display, where the position of thefirst display relative to the second display is determined using animage sensor.

In some embodiments, a computer-readable storage medium comprises one ormore programs for execution by one or more processors of a system havinga first electronic device with a first display and a second electronicdevice with a second display. The one or more programs includeinstructions which, when executed by the one or more processors, causethe system to: produce output data responsive to input; determine if thefirst device is operating in a first mode or a second mode; if the firstdevice is operating in the first mode, display content on the firstdisplay in accordance with the output data; if the first device isoperating in the second mode, refrain from displaying content on thefirst display in accordance with the output data, and transmit theoutput data to a second device; determine, using an image sensor, aposition of the first display relative to the second display; anddisplay, on the second display, content in accordance with the outputdata and the determined position of the first display relative to thesecond display.

In some embodiments, a method includes: at a first device coupled with afirst display and an image sensor: receiving output data from a seconddevice having a second display different from the first display, wherethe output data represents content displayable by the second device onthe second display; determining, using the image sensor, a position ofthe second display relative to the first device; and causing the firstdisplay to display content based on the output data received from thesecond device and the determined position of the second display relativeto the first device.

In some embodiments, an electronic device includes: one or moreprocessors; and memory storing one or more programs configured to beexecuted by the one or more processors. The one or more programs includeinstructions for: receiving output data from a second device having asecond display different from a first display coupled with theelectronic device, where the output data represents content displayableby the second device on the second display; determining, using an imagesensor coupled with the electronic device, a position of the seconddisplay relative to the electronic device; and causing the first displayto display content based on the output data received from the seconddevice and the determined position of the second display relative to theelectronic device.

In some embodiments, a computer-readable storage medium comprises one ormore programs for execution by one or more processors of an electronicdevice coupled with a first display and an image sensor. The one or moreprograms include instructions which, when executed by the one or moreprocessors, cause the electronic device to: receive output data from asecond device having a second display different from the first display,wherein the output data represents content displayable by the seconddevice on the second display; determine, using the image sensor, aposition of the second display relative to the electronic device; andcause the first display to display content based on the output datareceived from the second device and the determined position of thesecond display relative to the electronic device.

Executable instructions for performing these functions are, optionally,included in a non-transitory computer-readable storage medium or othercomputer program product configured for execution by one or moreprocessors. Executable instructions for performing these functions are,optionally, included in a transitory computer-readable storage medium orother computer program product configured for execution by one or moreprocessors.

Thus, devices are provided with more efficient and effective methods andinterfaces for providing a privacy screen on an electronic device,thereby protecting the content being produced and increasing theenvironments in which the device may be safely operated. Such methodsand interfaces may complement or replace other methods for safeguardingcontent while operating a device.

DESCRIPTION OF THE FIGURES

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIGS. 1A and 1B depict an exemplary system for use in various realitytechnologies, including computer simulated reality.

FIGS. 2A and 2B illustrate an exemplary electronic device and userinterfaces in accordance with some embodiments.

FIGS. 3A-3D illustrate exemplary electronic devices and user interfacesin accordance with some embodiments.

FIG. 4 illustrates exemplary electronic devices and user interfaces inaccordance with some embodiments.

FIG. 5 is a flow diagram illustrating a method for providing a privacyscreen on an electronic device.

FIG. 6 is a flow diagram illustrating a method for providing a privacyscreen on an electronic device.

DESCRIPTION OF EMBODIMENTS

The following description sets forth exemplary methods, parameters, andthe like. However, such description is not intended as a limitation onthe scope of the present disclosure but is instead provided as adescription of exemplary embodiments.

In the following description, a physical setting refers to a world thatindividuals can sense and/or with which individuals can interact withoutassistance of electronic systems. Physical settings (e.g., a physicalforest) include physical elements (e.g., physical trees, physicalstructures, and physical animals). Individuals can directly interactwith and/or sense the physical setting, such as through touch, sight,smell, hearing, and taste.

In contrast, a simulated reality (SR) setting refers to an entirely orpartly computer-created setting that individuals can sense and/or withwhich individuals can interact via an electronic system. In SR, a subsetof an individual's movements is monitored, and, responsive thereto, oneor more attributes of one or more virtual objects in the SR setting ischanged in a manner that conforms with one or more physical laws. Forexample, a SR system may detect an individual walking a few pacesforward and, responsive thereto, adjust graphics and audio presented tothe individual in a manner similar to how such scenery and sounds wouldchange in a physical setting. Modifications to attribute(s) of virtualobject(s) in a SR setting also may be made responsive to representationsof movement (e.g., audio instructions).

An individual may interact with and/or sense a SR object using any oneof his senses, including touch, smell, sight, taste, and sound. Forexample, an individual may interact with and/or sense aural objects thatcreate a multi-dimensional (e.g., three dimensional) or spatial auralsetting, and/or enable aural transparency. Multi-dimensional or spatialaural settings provide an individual with a perception of discrete auralsources in multi-dimensional space. Aural transparency selectivelyincorporates sounds from the physical setting, either with or withoutcomputer-created audio. In some SR settings, an individual may interactwith and/or sense only aural objects.

One example of SR is virtual reality (VR). A VR setting refers to asimulated setting that is designed only to include computer-createdsensory inputs for at least one of the senses. A VR setting includesmultiple virtual objects with which an individual may interact and/orsense. An individual may interact and/or sense virtual objects in the VRsetting through a simulation of a subset of the individual's actionswithin the computer-created setting, and/or through a simulation of theindividual or his presence within the computer-created setting.

Another example of SR is mixed reality (MR). A MR setting refers to asimulated setting that is designed to integrate computer-created sensoryinputs (e.g., virtual objects) with sensory inputs from the physicalsetting, or a representation thereof. On a reality spectrum, a mixedreality setting is between, and does not include, a VR setting at oneend and an entirely physical setting at the other end.

In some MR settings, computer-created sensory inputs may adapt tochanges in sensory inputs from the physical setting. Also, someelectronic systems for presenting MR settings may monitor orientationand/or location with respect to the physical setting to enableinteraction between virtual objects and real objects (which are physicalelements from the physical setting or representations thereof). Forexample, a system may monitor movements so that a virtual plant appearsstationary with respect to a physical building.

One example of mixed reality is augmented reality (AR). An AR settingrefers to a simulated setting in which at least one virtual object issuperimposed over a physical setting, or a representation thereof. Forexample, an electronic system may have an opaque display and at leastone imaging sensor for capturing images or video of the physicalsetting, which are representations of the physical setting. The systemcombines the images or video with virtual objects, and displays thecombination on the opaque display. An individual, using the system,views the physical setting indirectly via the images or video of thephysical setting, and observes the virtual objects superimposed over thephysical setting. When a system uses image sensor(s) to capture imagesof the physical setting, and presents the AR setting on the opaquedisplay using those images, the displayed images are called a videopass-through. Alternatively, an electronic system for displaying an ARsetting may have a transparent or semi-transparent display through whichan individual may view the physical setting directly. The system maydisplay virtual objects on the transparent or semi-transparent display,so that an individual, using the system, observes the virtual objectssuperimposed over the physical setting. In another example, a system maycomprise a projection system that projects virtual objects into thephysical setting. The virtual objects may be projected, for example, ona physical surface or as a holograph, so that an individual, using thesystem, observes the virtual objects superimposed over the physicalsetting.

An augmented reality setting also may refer to a simulated setting inwhich a representation of a physical setting is altered bycomputer-created sensory information. For example, a portion of arepresentation of a physical setting may be graphically altered (e.g.,enlarged), such that the altered portion may still be representative ofbut not a faithfully-reproduced version of the originally capturedimage(s). As another example, in providing video pass-through, a systemmay alter at least one of the sensor images to impose a particularviewpoint different than the viewpoint captured by the image sensor(s).As an additional example, a representation of a physical setting may bealtered by graphically obscuring or excluding portions thereof.

Another example of mixed reality is augmented virtuality (AV). An AVsetting refers to a simulated setting in which a computer-created orvirtual setting incorporates at least one sensory input from thephysical setting. The sensory input(s) from the physical setting may berepresentations of at least one characteristic of the physical setting.For example, a virtual object may assume a color of a physical elementcaptured by imaging sensor(s). In another example, a virtual object mayexhibit characteristics consistent with actual weather conditions in thephysical setting, as identified via imaging, weather-related sensors,and/or online weather data. In yet another example, an augmented realityforest may have virtual trees and structures, but the animals may havefeatures that are accurately reproduced from images taken of physicalanimals.

Many electronic systems enable an individual to interact with and/orsense various SR settings. One example includes head mounted systems. Ahead mounted system may have an opaque display and speaker(s).Alternatively, a head mounted system may be designed to receive anexternal display (e.g., a smartphone). The head mounted system may haveimaging sensor(s) and/or microphones for taking images/video and/orcapturing audio of the physical setting, respectively. A head mountedsystem also may have a transparent or semi-transparent display. Thetransparent or semi-transparent display may incorporate a substratethrough which light representative of images is directed to anindividual's eyes. The display may incorporate LEDs, OLEDs, a digitallight projector, a laser scanning light source, liquid crystal onsilicon, or any combination of these technologies. The substrate throughwhich the light is transmitted may be a light waveguide, opticalcombiner, optical reflector, holographic substrate, or any combinationof these substrates. In one embodiment, the transparent orsemi-transparent display may transition selectively between an opaquestate and a transparent or semi-transparent state. In another example,the electronic system may be a projection-based system. Aprojection-based system may use retinal projection to project imagesonto an individual's retina. Alternatively, a projection system also mayproject virtual objects into a physical setting (e.g., onto a physicalsurface or as a holograph). Other examples of SR systems include headsup displays, automotive windshields with the ability to displaygraphics, windows with the ability to display graphics, lenses with theability to display graphics, headphones or earphones, speakerarrangements, input mechanisms (e.g., controllers having or not havinghaptic feedback), tablets, smartphones, and desktop or laptop computers.

FIGS. 1A and 1B depict exemplary system 100 for use in various realitytechnologies, including computer simulated reality.

In some embodiments, as illustrated in FIG. 1A, system 100 includesdevice 100 a. Device 100 a includes various components, such asprocessor(s) 102, RF circuitry(ies) 104, memory(ies) 106, imagesensor(s) 108, orientation sensor(s) 110, microphone(s) 112, locationsensor(s) 116, speaker(s) 118, display(s) 120, and touch-sensitivesurface(s) 122. These components optionally communicate overcommunication bus(es) 150 of device 100 a.

In some embodiments, elements of system 100 are implemented in a basestation device (e.g., a computing device, such as a remote server,mobile device, or laptop) and other elements of the system 100 areimplemented in a second device (e.g., a head-mounted device). In someexamples, device 100 a is implemented in a base station device or asecond device.

As illustrated in FIG. 1B, in some embodiments, system 100 includes two(or more) devices in communication, such as through a wired connectionor a wireless connection. First device 100 b (e.g., a base stationdevice) includes processor(s) 102, RF circuitry(ies) 104, memory(ies)106. These components optionally communicate over communication bus(es)150 of device 100 b. Second device 100 c (e.g., a head-mounted device)includes various components, such as processor(s) 102, RF circuitry(ies)104, memory(ies) 106, image sensor(s) 108, orientation sensor(s) 110,microphone(s) 112, location sensor(s) 116, speaker(s) 118, display(s)120, and touch-sensitive surface(s) 122. These components optionallycommunicate over communication bus(es) 150 of device 100 c.

System 100 includes processor(s) 102 and memory(ies) 106. Processor(s)102 include one or more general processors, one or more graphicsprocessors, and/or one or more digital signal processors. In someembodiments, memory(ies) 106 are one or more non-transitorycomputer-readable storage mediums (e.g., flash memory, random accessmemory) that store computer-readable instructions configured to beexecuted by processor(s) 102 to perform the techniques described below.

System 100 includes RF circuitry(ies) 104. RF circuitry(ies) 104optionally include circuitry for communicating with electronic devices,networks, such as the Internet, intranets, and/or a wireless network,such as cellular networks and wireless local area networks (LANs). RFcircuitry(ies) 104 optionally includes circuitry for communicating usingnear-field communication and/or short-range communication, such asBluetooth®.

System 100 includes display(s) 120. Display(s) 120 may have an opaquedisplay. Display(s) 120 may have a transparent or semi-transparentdisplay that may incorporate a substrate through which lightrepresentative of images is directed to an individual's eyes. Display(s)120 may incorporate LEDs, OLEDs, a digital light projector, a laserscanning light source, liquid crystal on silicon, or any combination ofthese technologies. The substrate through which the light is transmittedmay be a light waveguide, optical combiner, optical reflector,holographic substrate, or any combination of these substrates. In oneembodiment, the transparent or semi-transparent display may transitionselectively between an opaque state and a transparent orsemi-transparent state. Other examples of display(s) 120 include headsup displays, automotive windshields with the ability to displaygraphics, windows with the ability to display graphics, lenses with theability to display graphics, tablets, smartphones, and desktop or laptopcomputers. Alternatively, system 100 may be designed to receive anexternal display (e.g., a smartphone). In some embodiments, system 100is a projection-based system that uses retinal projection to projectimages onto an individual's retina or projects virtual objects into aphysical setting (e.g., onto a physical surface or as a holograph).

In some embodiments, system 100 includes touch-sensitive surface(s) 122for receiving user inputs, such as tap inputs and swipe inputs. In someexamples, display(s) 120 and touch-sensitive surface(s) 122 formtouch-sensitive display(s).

System 100 includes image sensor(s) 108. Image sensors(s) 108 optionallyinclude one or more visible light image sensor, such as charged coupleddevice (CCD) sensors, and/or complementary metal-oxide-semiconductor(CMOS) sensors operable to obtain images of physical elements from aphysical setting. Image sensor(s) also optionally include one or moreinfrared (IR) sensor(s), such as a passive IR sensor or an active IRsensor, for detecting infrared light from the physical setting. Forexample, an active IR sensor includes an IR emitter, such as an IR dotemitter, for emitting infrared light into the physical setting. Imagesensor(s) 108 also optionally include one or more event camera(s)configured to capture movement of physical elements in the physicalsetting. Image sensor(s) 108 also optionally include one or more depthsensor(s) configured to detect the distance of physical elements fromsystem 100. In some examples, system 100 uses CCD sensors, eventcameras, and depth sensors in combination to detect the physical settingaround system 100. In some examples, image sensor(s) 108 include a firstimage sensor and a second image sensor. The first image sensor and thesecond image sensor are optionally configured to capture images ofphysical elements in the physical setting from two distinctperspectives. In some examples, system 100 uses image sensor(s) 108 toreceive user inputs, such as hand gestures. In some examples, system 100uses image sensor(s) 108 to detect the position and orientation ofsystem 100 and/or display(s) 120 in the physical setting. For example,system 100 uses image sensor(s) 108 to track the position andorientation of display(s) 120 relative to one or more fixed elements inthe physical setting.

In some embodiments, system 100 includes microphones(s) 112. System 100uses microphone(s) 112 to detect sound from the user and/or the physicalsetting of the user. In some examples, microphone(s) 112 includes anarray of microphones (including a plurality of microphones) thatoptionally operate in tandem, such as to identify ambient noise or tolocate the source of sound in space of the physical setting.

System 100 includes orientation sensor(s) 110 for detecting orientationand/or movement of system 100 and/or display(s) 120. For example, system100 uses orientation sensor(s) 110 to track changes in the positionand/or orientation of system 100 and/or display(s) 120, such as withrespect to physical elements in the physical setting. Orientationsensor(s) 110 optionally include one or more gyroscopes and/or one ormore accelerometers.

Turning now to FIGS. 2A-2B, 3A-3D, and 4 , exemplary electronic devicesand user interfaces for providing a privacy screen on an electronicdevice (e.g., 100 a), in accordance with some embodiments, aredescribed. The figures are used to illustrate the processes describedbelow, including the processes in FIGS. 5 and 6 .

FIG. 2A illustrates an electronic device 200 with a display 202 thatpresents content (e.g., a graphical user interface (GUI) associated withthe state of device 200). In FIG. 2A, the content displayed by device200 includes GUI screen 204 with selectable icons for launchingrespective applications. In some embodiments, device 200 is asmartphone, tablet computer, laptop computer, desktop computer,smartwatch, or other electronic device that has a display. In someembodiments, device 200 is capable of receiving inputs via one or moreof a keyboard, mouse, joystick, touch-sensitive surface (e.g., atouch-pad or touch-sensitive display), button, rotatable input mechanism(e.g., a scroll wheel), speaker (e.g., for voice inputs or commands), orother input mechanism. In the illustrated embodiment, display 202 is atouch-sensitive display.

Device 200 is configured to receive an input (e.g., a contact ontouch-sensitive display 202) and produce output data responsive to theinput. In some embodiments, the output data represents content (e.g.,images and/or video) that is displayable by device 200 on display 202.In some embodiments, the aspect ratio of the content (e.g., GUI screen204) is based on the aspect ratio of display 202.

Device 200 is also configured to determine if it is operating in anormal mode or a private mode. FIGS. 2A and 2B illustrate the operationof device 200 in a normal mode. Device 200 receives touch input 210 onicon 206. In response, device 200 launches a corresponding mailapplication and produces output data representing a GUI screen for themail application. As device 200 is operating in the normal mode, itdisplays GUI screen 208 for the mail application on display 202 inaccordance with the output data.

FIGS. 3A-3D illustrate device 200 operating in a private mode and anexternal device 300 (e.g., a tablet), with display 302, that isconfigured to communicate with device 200. In the illustratedembodiment, external device 300 supports AR functionality (e.g., usingvideo pass-through) and display 302 superimposes virtual objects overimages of a physical setting. The depictions on the left side of FIGS.3A-3D illustrate an unobstructed view of device 200 being held in thehand of a user. The depictions on the right side of FIGS. 3A-3Dillustrate the view of device 200 being held in the hand of the user asseen on display 302 of external device 300, meaning that a camera ofdevice 300 is aimed at device 200 while device 200 is operating inprivate mode.

In private mode, device 200 is configured to not display content ondisplay 202 (or display a reduced set of content that hides sensitiveinformation) while otherwise operating normally. In some embodiments,the private mode is activated in response to a user input (e.g., a userinput at device 200) representing a request to enter private mode. Insome embodiments, device 200 activates private mode when it determinesthat device 200 is being viewed using an AR device such as externaldevice 300.

In FIG. 3A, device 200 is operating in the same state illustrated inFIG. 2A, which provides an interface for launching applications, butdoes not display GUI screen 204 on display 202 because the device is inprivate mode. Instead, device 200 transmits the output data to externaldevice 300. In some embodiments, device 200 transmits the output data toexternal device 300 via a wired and/or wireless communication link(e.g., via Bluetooth®, Near-field Communication (NFC), WiFi, or otherwireless communication protocol).

External device 300 is configured to display content on display 302 inaccordance with the output data transmitted by device 200. As shown inFIG. 3A, external device 300 displays GUI screen 204. In someembodiments, external device 300 sends confirmation to device 200 that acommunication link has been established (e.g., in response toestablishing a communication link with device 200) and/or that thecontent is being successfully displayed on display 302 (e.g., inresponse to successfully receiving output data from device 200 or inresponse to successfully displaying output data received from device200). Optionally, if device 200 is operating in a private mode and doesnot receive confirmation that a communication link has been establishedor determines that the communication link is poor or has beeninterrupted, device 200 and/or external device 300 provides anindication that the communication link is poor (e.g., beeps or displaysa message, such as “Connection is poor”), while device 200 remains inprivate mode to maintain the protection of the content on device 200. Ifthe connection is poor, device 200 optionally displays a limitedinterface to allow a user to exit private mode (e.g., a simple menu toexit private mode that does not include potentially sensitive content).

In the embodiment illustrated in FIGS. 3A-3D, although device 200 doesnot display content, it still provides an interface for performingoperations while in private mode. For example, while in private mode,device 200 receives touch input 310 (shown in FIG. 3B) at the samelocation on display 202 as icon 206 illustrated in FIG. 2A. In responseto touch input 210, device 200 launches the mail application andproduces output data representing a GUI screen for the mail application.Since device 200 is operating in the private mode, device 200 refrainsfrom displaying content (e.g., GUI screen 208 for the mail application)on display 202 and transmits the output data to external device 300.External device 300 receives the output data produced in response toinput 310 and displays the GUI screen 208 on display 302, as shown inFIG. 3C. Thus, the content displayed by external device 300 is inaccordance with the operation performed by device 200 (e.g., launchingthe email application) in response to the input detected by device 200(e.g., touch input 310). Further, the external device 300 updates thecontent displayed on display 202 in response to the input at device 200like device 200 would if device 200 were displaying the content itselfon display 202.

Accordingly, in the private mode, device 200 receives inputs andperforms operations as it would in a normal operational mode, but doesnot display the corresponding content. Instead, the content is displayedon a display 302 of external device 300. It is noted that, whileoperating in private mode, device 200 can refrain from energizing someor all of the light emitting diodes (e.g., turning off the display LEDs)of touchscreen display 202 to preserve battery life while retaining thetouch-detection functionality.

It should be recognized that FIGS. 3A-3D illustrate one exemplaryembodiment. In some embodiments, external device 300 is a head mountedsystem (e.g., a head-mounted device) that allows the user to see aphysical view of device 200 through (at least partially) transparentdisplays in combination with virtual object(s) (e.g., GUI screen 108)displayed by the head mounted system. In some embodiments, externaldevice 300 is a projection-based or other SR system that carries out acomputer simulated reality technique that includes some or all of thefeatures described above. In some embodiments, external device 300 isconnected by a wire or wirelessly (e.g., via Bluetooth® protocol) todisplay 302 (or another device that includes display 302). In someexamples, external device 300 is a computing device (e.g., ansmartphone) in communication with a head mounted system that includesdisplay 302. This may allow the techniques described herein to beimplemented with a display (e.g., a headset) that is configured tocommunicate directly with external device 300 but not with device 200.

When display 302 of external device 300 can only be viewed by the user(such as in the case of a personal headset worn by the user), thisfeature provides privacy by allowing the user to operate device 200while preventing other people from viewing the content (e.g., e-mails,documents, applications, etc.) being processed by device 200, whichwould typically be provided on display 202. For example, FIG. 4illustrates device 400 (e.g., a laptop computer) operating in privatemode with head mounted system 450. As shown in FIG. 4 , device 400 canbe seen through the display 452 of head mounted system 450 and is beingused to edit a confidential document that is only presented on display452. Since the content of the document is only displayed on display 452of head mounted system 450, device 400 can be used privately to edit thedocument, such as by device 400 receiving user inputs on a keyboardand/or touchpad of device 400.

Returning again to the devices depicted in FIGS. 3A-3D, in someembodiments, when specifically instructed by a user, device 200transmits the output data to more than one external device (e.g., toexternal device 300 and to another external device). For example, device200 transmits the output data to an external device operated by the userof device 200 and to an external device of other users so that theauthorized peers of the user of device 200 can view a presentationconcurrently. In this way, a user can share the content of device 200with those authorized by the user while maintaining the privacy ofdisplay 202 from those who are not authorized. In some embodiments, uponspecific user instruction, device 200 receives a first configurationrequest to enable viewing of content on multiple devices. In response toreceiving the first configuration request, device 200 transmits outputdata to a first external device and a second external device.Subsequently, device 200 receives a second configured request to disableviewing of content on multiple devices. In response to receiving thesecond configuration request, device 200 continues to transmit outputdata to the first external device and ceases transmitting output data tothe second external device.

In some embodiments, external device 300 determines the position (e.g.,location and/or orientation) of display 202 and displays the contentbased on the determined position of display 202. The position can be anabsolute position or a relative position (e.g., relative to the positionof display 302). In some embodiments, external device 300 determines theposition of display 202 based at least in part on position data receivedfrom device 200. For example, device 200 may include sensors (e.g.,accelerometers, gyroscopes, etc.) that determine a position of display202, which is then sent to external device 300. In some embodiments,external device 300 includes one or more sensors (e.g., an exteriorcamera) that determine a position of display 202 (e.g., using imageprocessing and/or feature recognition processing). External device 300may also include sensors to determine the position and/or motion of theexternal device 300 itself, which is then used to determine the positionof display 202 relative to display 302.

In some embodiments, the position of display 202 is used to determinethe position of the content on display 302. That is, the position of thecontent on display 302 is based on the position of display 202 (e.g.,the absolute position of display 202 or the position of display 202relative to display 302). For example, in the embodiment illustrated inFIGS. 3A-3D, the position of GUI screens 204 and 208 correspond to theposition of display 202 such that GUI screens 204 and 108 aresuperimposed on display 202 of device 200. In other words, GUI screens204 and 208 appear to the user as though they are displayed on device200, even though they are actually displayed on display 302 of externaldevice 300. Thus, when viewed using external device 300, device 200appears as it would in normal mode. This allows the user to easilyoperate device 200. For example, since the icons are not actuallydisplayed on display 202, using display 302 to superimpose the GUIscreen 204 on display 202 assists the user in touching the correct areaof the display for performing the desired operation.

In some embodiments, device 200 further determines an updated positionof display 202 and then changes the position of the content on display302 based on the updated position. In some examples, device 200 tracksthe position of display 202 and updates the displayed position of thecontent so that the content appears to be maintained on display 202 asdisplay 202 moves. In some embodiments, in order to maintain theappearance that the content is displayed on display 202, external device300 transforms the output data from device 200 (e.g., translates,rotates, enlarges, shrinks, stretches, compresses, or otherwisemanipulates the image to be displayed) such that the content appears asthough it is displayed on display 202. For example, as shown in FIG. 3D,display 202 is rotated compared to the position shown in FIGS. 3A-3C. Tocompensate for the change in position, external device 300 compressesthe image of GUI screen 208 and applies a perspective effect so that theimage appears to be mapped onto display 202, as seen from the point ofview of the user.

FIG. 5 is a flow diagram illustrating a method for providing a privacyscreen on an electronic device in accordance with some embodiments.Method 500 is performed at system that includes a first device (e.g.,device 200) with a first display (e.g., display 202) and a second device(e.g., device 300) with a second display (e.g., display 302) differentthan the first display. In some embodiments, the first device is alaptop, tablet, smartphone, or smartwatch. In some embodiments, thesecond device is a head-mounted device. Some operations in method 500are, optionally, combined, the orders of some operations are,optionally, changed, and some operations are, optionally, omitted.

At block 502, the first device receives a first input (e.g., input 301).In some embodiments, the first display is a touch-screen display and thefirst input is a contact on the touch-screen. At block 504, the firstdevice produces output data responsive to the first input. In someembodiments, the output data includes visual output (e.g., video data).

At block 506, the first device determines if the first device isoperating in a first (e.g., normal) mode or a second (e.g., private)mode. At block 508, if the first device is operating in the first mode,the first device displays content on the first display in accordancewith the output data. Optionally, if the first device is operating inthe first mode, the first device refrains from transmitting the outputdata to the second device. At block 510, if the first device isoperating in the second mode, the first device refrains from displayingcontent on the first display in accordance with the output data, andtransmits the output data to the second device. Optionally, refrainingfrom displaying content on the first display in accordance with theoutput data includes not displaying any content on the first display,not presenting visible content on the first display, or turning offdisplay elements (e.g., LEDs) of the first display. Optionally, if thefirst device is operating in the second mode, the first device transmitsthe output data to a third device different from the first device andthe second device.

At block 512, the second device determines (e.g., using an image sensor)a position of the first display (e.g., relative to the second display).In some embodiments, determining the position of the first displayincludes obtaining an image of the first device with an image sensor ofthe second device and determining, using the obtained image, theposition of the first display (e.g., relative to the second display). Atblock 414, the second device displays, on the second display, content inaccordance with the output data and the determined position of the firstdisplay. Optionally, the position of the content on the second displayis based on the position of the second display. Optionally, the positionof the content on the second display is based on the position of thesecond device with respect to the first device. Optionally, the positionof the content on the second display corresponds to the position of thefirst display. In some embodiments, an image of the first device isobtained using an image sensor of the second device, and displaying thecontent includes displaying (e.g., on the second display) arepresentation of at least a portion of the first device using theobtained image. Optionally, the second device determines a secondposition of the first display (e.g., relative to the second display),and changes the position of the content on the second display based onthe determined second position of the first display.

FIG. 6 is a flow diagram illustrating a method for providing a privacyscreen on an electronic device in accordance with some embodiments.Method 600 is performed at a first device (e.g., device 300) coupledwith a first display (e.g., display 302) and, optionally, an imagesensor. In some embodiments, the first device is a head mounted systemthat supports simulated reality features. Some operations in method 600are, optionally, combined, and the orders of some operations are,optionally, changed. Some operations in method 600 are, optionally,included in method 500, and some operations in method 500 are,optionally, included in method 600.

At block 602, the first device receives output data from a second device(e.g., device 200) having a second display (e.g., display 202) differentfrom the first display. In some embodiments, the second device is alaptop, tablet, smartphone, or smartwatch. In some embodiments, theoutput is generated by the second device responsive to contact on atouch-screen display of the second device. Optionally, the output datarepresents content displayable by the second device on the seconddisplay. Optionally, the content includes visual output (e.g., videodata).

At block 604, the first device determines (e.g., using an image sensor)a position of the second display (e.g., relative to the first device).At block 506, the first device causes the first display to displaycontent based on the output data received from the second device and thedetermined position of the second display. In some embodiments, an imageof the second device is obtained (e.g., using an image sensor of thefirst device), and causing the first display to display content includescausing display of a representation of at least a portion of the firstdevice using the obtained image. Optionally, the position of the content(e.g., display location of the content on the first display) is based onthe position of the second display. Optionally, the position of thecontent (e.g., display location of the content on the first display)corresponds to the position of the second display. Optionally, the firstdevice determines a second position of the second display (e.g.,relative to the first display) and changes the position of the contentbased on the determined second position of the second display.Optionally, the first device updates the content displayed at the firstdisplay responsive to an input (e.g., input 310) at the second device.Optionally, the updated content is in accordance with an operationperformed by the second device in response to the input detected by thesecond device.

In some embodiments, the content is not displayed by the second deviceon the second display while the first display is displaying the content.Optionally, the first device sends instructions to the second device torefrain from displaying the content on the second display or frompresenting visible content on the second display. Optionally, theinstructions include instructions to turn off elements of the seconddisplay. In some embodiments, the first device sends confirmation to thesecond device that a communication link between the first device and thesecond device has been established. Optionally, the first deviceprovides an indication in response to the communication link beinginterrupted. In some embodiments, the first device causes the content tobe displayed on a third display different from the first display and thesecond display.

Executable instructions for performing the features of methods 500and/or 600 described above are, optionally, included in a transitory ornon-transitory computer-readable storage medium (e.g., memory(ies) 106)or other computer program product configured for execution by one ormore processors (e.g., processor(s) 102).

Aspects of the techniques described above contemplate the possibility ofgathering and using personal information to provide an improved privacyscreen on an electronic device. Such information should be collectedwith the user's informed consent.

Entities handling such personal information will comply withwell-established privacy practices and/or privacy policies (e.g., thatare certified by a third-party) that are (1) generally recognized asmeeting or exceeding industry or governmental requirements, (2)user-accessible, (3) updated as needed, and (4) compliant withapplicable laws. Entities handling such personal information will usethe information for reasonable and legitimate uses, without sharing orselling outside of those legitimate uses.

However, users may selectively restrict access/use of personalinformation. For example, users can opt into or out of collection oftheir personal information. In addition, although aspects of thetechniques described above contemplate use of personal information,aspects of the techniques can be implemented without requiring or usingpersonal information. For example, if location information, usernames,and/or addresses are gathered, they can be generalized and/or masked sothat they do not uniquely identify an individual.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the techniques and their practical applications. Othersskilled in the art are thereby enabled to best utilize the techniquesand various embodiments with various modifications as are suited to theparticular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims.

What is claimed is:
 1. A first electronic device, comprising: a firstdisplay; one or more processors; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: producing visual output data; whilethe first electronic device is operating in a first mode, displayingcontent on the first display corresponding to the visual output data;receiving instructions from a second electronic device to refrain fromdisplaying content on the first display; after receiving theinstructions from the second electronic device, activating a second modeof the first electronic device; while the first electronic device isoperating in the second mode: refraining from displaying content on thefirst display corresponding to the visual output data; and transmittingthe visual output data from the first electronic device to the secondelectronic device, wherein the content corresponding to the visualoutput data is displayable by the second electronic device on a seconddisplay.
 2. The first electronic device of claim 1, wherein theinstructions from the second electronic device to refrain fromdisplaying content on the first display includes instructions to turnoff display elements of the first display.
 3. The first electronicdevice of claim 1, wherein the one or more programs further includeinstructions for: after receiving the instructions from the secondelectronic device, sending a confirmation to the second device that acommunication link between the first electronic device and the secondelectronic device has been established.
 4. The first electronic deviceof claim 1, wherein the one or more programs further includeinstructions for: updating the content corresponding to the visualoutput data in response to an input at the first electronic device. 5.The first electronic device of claim 1, wherein the first electronicdevice includes a touch-screen display and wherein the visual outputdata is produced by the first electronic device in response to contacton the touch-screen display.
 6. A non-transitory computer-readablestorage medium storing one or more programs configured to be executed byone or more processors of a first electronic device with a firstdisplay, the one or more programs including instructions for: producingvisual output data; while the first electronic device is operating in afirst mode, displaying content on the first display corresponding to thevisual output data; receiving instructions from a second electronicdevice to refrain from displaying content on the first display; afterreceiving the instructions from the second electronic device, activatinga second mode of the first electronic device; while the first electronicdevice is operating in the second mode: refraining from displayingcontent on the first display corresponding to the visual output data;and transmitting the visual output data from the first electronic deviceto the second electronic device, wherein the content corresponding tothe visual output data is displayable by the second electronic device ona second display.
 7. The non-transitory computer-readable storage mediumof claim 6, wherein the one or more programs further includeinstructions for: updating the content corresponding to the visualoutput data in response to an input at the first electronic device. 8.The non-transitory computer-readable storage medium of claim 6, whereinthe first electronic device includes a touch-screen display and whereinthe visual output data is produced by the first electronic device inresponse to contact on the touch-screen display.
 9. A method,comprising: at a first electronic device with a first display: producingvisual output data; while the first electronic device is operating in afirst mode, displaying content on the first display corresponding to thevisual output data; receiving instructions from a second electronicdevice to refrain from displaying content on the first display; afterreceiving the instructions from the second electronic device, activatinga second mode of the first electronic device; while the first electronicdevice is operating in the second mode: refraining from displayingcontent on the first display corresponding to the visual output data;and transmitting the visual output data from the first electronic deviceto the second electronic device, wherein the content corresponding tothe visual output data is displayable by the second electronic device ona second display.
 10. The method of claim 9, further comprising:updating the content corresponding to the visual output data in responseto an input at the first electronic device.
 11. The method of claim 9,wherein the first electronic device includes a touch-screen display andwherein the visual output data is produced by the first electronicdevice in response to contact on the touch-screen display.
 12. A secondelectronic device, comprising: a second display; one or more processors;and memory storing one or more programs configured to be executed by theone or more processors, the one or more programs including instructionsfor: obtaining, using an image sensor, an image of a first electronicdevice that includes a first display; transmitting instructions to thefirst electronic device to refrain from displaying content on the firstdisplay; after transmitting the instructions to the first electronicdevice, receiving visual output data from the first electronic device,wherein the visual output data was produced by the first electronicdevice for display on the first display; and displaying, on the seconddisplay, content corresponding to the visual output data.
 13. The secondelectronic device of claim 12, wherein the one or more programs furtherinclude instructions for: displaying, on the second display, the imageof the first electronic device obtained with the image sensor, whereinthe image of the first electronic device is concurrently displayed withthe content corresponding to the visual output data.
 14. The secondelectronic device of claim 13, wherein the content corresponding to thevisual output data is displayed overlaying the first display of thefirst electronic device.
 15. The second electronic device of claim 12,wherein the one or more programs further include instructions for:determining a position of the first display relative to the seconddisplay, wherein the content is displayed on the second display at alocation on the second display based in part on the position of thefirst display relative to the second display.
 16. The second electronicdevice of claim 12, wherein the second electronic device is ahead-mounted device.
 17. A non-transitory computer-readable storagemedium storing one or more programs configured to be executed by one ormore processors of a second electronic device with a second display, theone or more programs including instructions for: obtaining, using animage sensor, an image of a first electronic device that includes afirst display; transmitting instructions to the first electronic deviceto refrain from displaying content on the first display; aftertransmitting the instructions to the first electronic device, receivingvisual output data from the first electronic device, wherein the visualoutput data was produced by the first electronic device for display onthe first display; and displaying, on the second display, contentcorresponding to the visual output data.
 18. The non-transitorycomputer-readable storage medium of claim 17, wherein the one or moreprograms further include instructions for: displaying, on the seconddisplay, the image of the first electronic device obtained with theimage sensor, wherein the image of the first electronic device isconcurrently displayed with the content corresponding to the visualoutput data.
 19. The non-transitory computer-readable storage medium ofclaim 18, wherein the content corresponding to the visual output data isdisplayed overlaying the first display of the first electronic device.20. The non-transitory computer-readable storage medium of claim 17,wherein the one or more programs further include instructions for:determining a position of the first display relative to the seconddisplay, wherein the content is displayed on the second display at alocation on the second display based in part on the position of thefirst display relative to the second display.
 21. A method, comprising:at a second electronic device with a second display: obtaining, using animage sensor, an image of a first electronic device that includes afirst display; transmitting instructions to the first electronic deviceto refrain from displaying content on the first display; aftertransmitting the instructions to the first electronic device, receivingvisual output data from the first electronic device, wherein the visualoutput data was produced by the first electronic device for display onthe first display; and displaying, on the second display, contentcorresponding to the visual output data.
 22. The method of claim 21,further comprising: displaying, on the second display, the image of thefirst electronic device obtained with the image sensor, wherein theimage of the first electronic device is concurrently displayed with thecontent corresponding to the visual output data.
 23. The method of claim22, wherein the content corresponding to the visual output data isdisplayed overlaying the first display of the first electronic device.24. The method of claim 21, further comprising: determining a positionof the first display relative to the second display, wherein the contentis displayed on the second display at a location on the second displaybased in part on the position of the first display relative to thesecond display.